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DETAILED ACTION/ 

1 . This Final action is responsive to the amendment filed 4/30/2008 and IDS filed 4/25/2008. 

2. In the amendment Claims 1-21 are pending. Claims 1, 9, 15 and 19 are the independent claims. 

Information Disclosure Statement 

3. The information disclosure statement (IDS) submitted on 4/30/2008 has been entered, and considered by the examiner. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation ol'35 U.S.C. 103(a) which forms the basis for all ob\ iotisness rejections set forth in this 
Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill 
in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

5. Claims 1-21 remain rejected under 35 U.S.C. 103(a) as being unpatentable over Purnaveja (U.S. 6,006,241, filed on 
Mar. 14, 1997) in view of Sani (NPL— Java Applets #4, April 1998, irt.org, pgs 1-6). 

Regarding Independent claim 1, A method of displaying a markup document and a linked applet within the markup 
document, the method comprising: I )ela> ing display of image output information for the markup document using 
image output delay information used to delay display of the markup document, and included in the applet or the 
markup document; and Synchronizing the delayed image output information for the markup document with applet 
output information lor the applet linked to the markup document, when rendering of the applet is completed, such that 
the delayed image output information for the markup document and the applet output information for the applet arc 
displayed simultaneously, wherein the applet is formed using the Java programming language. 

Purnaveja teaches the synchronized display of textual/graphical HTML data with Java applets (see abstract & column 
2, lines 40-55). He teaches the dela\ of markup image audio data by using a decoder which buffers the markup data 
(see fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58). He states that "Further, since the video 
and annotation streams are handled synchronously but separately by video decoder 964 and annotation interpreter 
963, respectively, steps 1040 and 1050 can occur concurrently. . "(see column '•■).. lines 34-40). Although Purnaveja 
provides a reasonable suggestion for simultaneous display of HTML data with applets he fails to teach the use of delay 
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functions within the applet lor delay ing applet duly, instead rely ing on the interpreter decoder to display both data 
concurrently further using time markers to display data at a predetermined time, thus including attributes to delay 
markup data display . Sani leaches the use of common Java Applet functions that include delay suspending of applet 
data (see pg 2, paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2). He also describes the common applet 
execution states for start, stop and destroy operations using functions (see pg 2, paragraphs 1 & 3). At the time of the 
invention it would have been obvious for the skilled artisan to ha\e implemented the common delay suspend Java 
applet functions of Sani with the invention of Purnaveja. The moth at ion for doing so would have been to continuously 
display HTML and applet data concurrently thereby providing information in an integrated seamless package to client 
computers. 

Regarding Dependent claim 2, with dependency of claim 1, Purnaveja teaches w herein the delaying of the display of 
the image output information for the markup document comprises buffering the image output information for the 
markup document (see abstract & column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically 
lines 34-58, including the explanation provided in the independent claims). 

Regarding Dependent claim 3, with dependency of claim 1, Purnaveja teaches wherein the synchronously displaying 
the delayed image output information lor the markup document and the applet output for an initial image of the applet 
comprises simultaneously providing the delayed image output information for the markup document and the applet 
output for the initial image of the applet to a display device based on an output control signal (see abstract & column 2, 
lines 40-55 & tig 9 numerals 466-465 & column 4, lines 5-65, specifically lines 34-5X, including the explanation 
provided in the independent claims). 

Regarding Dependent claim 4, with dependency of claim 1, Purnaveja teaches wherein the applet is formed of 
program codes having an output method different from that of the markup document (see abstract & column 2, lines 
40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the explanation provided in 
the independent claims). 

Regarding Dependent claim 5, with dependency of claim 3, l'urna\ eja teaches wherein the output control signal is 
provided from an applet executing engine, which interprets the applet, or a presentation engine, which interprets the 
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markup document (see abstract & column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically 
lines 34-58. including the explanation pro\ ided in the independent claims). 

Regarding Dependent claim 6, with dependency of claim 1, Purnaveja teaches wherein the delaying of the display of 
the image output information for the markup document comprises buffering text output of the markup document and 
buffering at least one of an image output and an audio output of the markup document (see abstract & column 2, lines 
40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the explanation provided in 
the independent claims). 

Regarding Dependent claim 7, with dependency of claim 2, Purnaveja teaches wherein the buffering comprises 
buffering text output of the markup document and buffering at least one of an image output and an audio output of the 
markup document (see abstract & column 2, lines 40-55 & lig 4 numerals 466-465 & column 9, lines 5-65, specifically 
lines 34-58, including the explanation provided in the independent claims). 

Regarding Dependent claim 8, with dependency of claim 3, Purnaveja teaches wherein the delaying of the display of 
the image output information for the markup document comprises buffering text output of the markup document and 
buffering at least one of an image output and an audio output of the markup document (see abstract & column 2, lines 
40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the explanation provided in 
the independent claims). 

Regarding Independent claim 9, An information storage medium encoded with compute!- an d or pr< >ccssor-executable 
instructions used to control a computer to display a markup document and a linked applet within the markup document, 
comprising: the markup document; and the applet linked to the markup document, wherein the applet or the markup 
document includes markup image output delay information used to delay display of the markup document such that 
image output information of the markup document and applet output information of the applet are synchronized to be 
displayed simultaneously, wherein the applet is formed using the Java programming language. 

Purnaveja teaches the synchronized display of textual graphical HTML data with Java applets ( see abstract & column 
2, lines 40-55). He teaches the delay of markup image/audio data by using a decoder which buffers the markup data 
(see fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58). He states that "Further, since the video 
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and annotation streams arc handled synchronously but separately by video decoder 964 and annotation interpreter 
963, respectively, steps 1040 and 1050 can occur concurrently... "(see column 9, lines 34-40). Although Purnaveja 
provides a reasonable suggestu m lor simultaneous display of HTML data with applets he fails to teach the use of delay 
functions within the applet lor delaying applet data, instead rely ing on the interpreter decoder to display both data 
concurrently further using time markers to display data at a predetermined time, thus including attributes to delay 
markup data display. Sani teaches the use of common Java Applet functions that include delay/ suspending of applet 
data (see pg 2, paragraphs 1 & 3 & pg 3, paragraphs 1 -2 & pg 4, paragraphs 1 -2 ). 1 le also describes the common applet 
execution states for start, stop and destroy operations using functions (see pg 2, paragraphs 1 & 3). At the time of the 
invention it would have been ob\ ions for the skilled artisan to ha\e implemented the common dela> suspend Ja\ a 
applet functions of Sani with the imention of l'urna\ eja. The moti\ation lor doing so would ha\e been to continuously 
display HTML and applet data concurrently thereby pi\>\ id ing information in an integrated seamless package to client 
computers. 

Regarding Dependent claim 10, with dependency of claim 9, Although Purnaveja provides a reasonable suggestion 
for simultaneous display of HTML data with applets he fails to teach the use of delay functions within the applet for 
delaying applet data, instead relying on the interpreter/decoder to display both data concurrently further using time 
markers to display data at a predetermined time, thus including attributes to delay markup data display. Sani teaches 
wherein the applet executes in any one state of an initial state, a start state, a stop state, and a destroy state (see pg 2, 
paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1 -2, including the explanation provided in the 
Independent claims). At the time of the invention it would have been ob\ ions lor the skilled artisan to have 
implemented the common delay suspend Ja\ a applet functions of Sani with the invention of Purnaveja. The motivation 
for doing so would have been to continuously display HTML and applet data concurrently thereby providing 
information in an integrated seamless package to client computers. 

Regarding Dependent claim 11, with dependency of claim 9, Although Purnax eja pro\ ides a reasonable suggestion 
for simultaneous display of HTML data with applets he fails to teach the use of delay functions within the applet for 
delaying applet data, instead relying on the interpreter/decoder to display both data concurrently further using time 
markers to display data at a predetermined time, thus including attributes to delay markup data display. Sani teaches 
wherein the applet includes a delay function as the markup image output delay information for synchronizing display of 
image output information of the markup document with display of output information of the applet ( see pg 2, 
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paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2, including the explanation provided in the 
Independent claims). At the time of the invention it would have been obvious for the skilled artisan to have 
implemented the common delay suspend Java applet functions of Sani with the invention of Purnaveja. The motivation 
for doing so would have been to continuously display HTML and applet data concurrently thereby providing 
information in an integrated seamless package to client computers. 

Regarding Dependent claim 12, with dependency of claim 10, Although Purnaveja pro\ ides a reasonable suggestion 
for simultaneous display of HTML data with applets he fails to teach the use of delay functions within the applet for 
delaying applet data, instead relying on the interpreter/ decoder to display both data concurrently further using time 
markers to display data at a predetermined time, thus including attributes to delay markup data display. Sani teaches 
wherein the applet includes a delay function during the start state as the markup image output delay information for 
synchronizing display of image output information of the markup document with display of output information of the 
applet (seepg 2, paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2, including the explanation provided 
in the Independent claims). At the time of the invention it would have been obvious for the skilled artisan to have 
implemented the common delay/suspend Java applet functions of Sani with the invention of Purnaveja. The motivation 
for doing so would have been to continuously display HTML and applet data concurrently thereby providing 
information in an integrated seamless package to client computers. 

Regarding Dependent claim 13, with dependency of claim 10, Although Purnaveja provides a reasonable suggestion 
for simultaneous display of HTML data with applets he fails to teach the use of delay functions within the applet for 
delaying applet data, instead relying on the interpreter/decoder to display both data concurrently further using time 
markers to display data at a predetermined time, thus including attributes to delay markup data display. Sani teaches 
wherein the applet comprises: a delay function as the markup image output delay information, which delays display of 
image output information for the markup document; and a delay cancel function canceling the delay of the display of 
the image output information for the markup document, when rendering of an initial image of the applet is completed 
by the initial and start slates of the applet (see pg 2, paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2, 
including the explanation pro\ ided in the Independent claims). At the time of the invention it would have been obvious 
for the skilled artisan to have implemented the common delay suspend Java applel functions of Sani with the invention 
of Purnaveja. The motivation for doing so would have been to continuously display HTML and applet data 
concurrently thereby providing information in an integrated seamless package to client computers. 
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Regarding Dependent claim 14, with dependency of claim 9, Pumaveja leaches wherein the markup document 
comprises tag or attribute indication information as the markup image output delay information to control synchronous 
display of output of the markup document with output of the applet (see abstract & column 2, lines 40-55 & fig 9 
numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the explanation provided in the 
independent claims). 

Regarding Independent claim 15, A computer system with a display device to display a markup document and a 
linked applet within the markup document, comprising: a presentation engine, which interprets the markup document to 
provide image output information for the markup document; 

and an applet executing engine, which interprets the applet linked to the markup document to provide an applet output, 
wherein the presentation engine delays display of the image output information for the markup document using image 
output delay information used to delay display of the markup document, and included in the applet or the markup 
document, and synchronizes and outputs the delayed image output information of the markup document and the applet 
output to the display device for simultaneous display, when an output control signal indicating completion of rendering 
of the applet output is input from the applet executing engine, and the applet is formed using the Java programming 
language, 

Pumaveja teaches the synchronized display ol textual graphical HTML data with Java applets (sec abstract & column 
2, lines 40-55). He teaches the delay of markup image audio data by using a decoder which buffers the markup data 
(see fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58). He states that "Further, since the video 
and annotation streams are handled synchronously but separately by video decoder 964 and annotation interpreter 
963, respectively, steps 1040 and 1050 can occur concurrently... "(see column 9, lines 34-40). Although Pumaveja 
provides a reasonable suggestion for simultaneous display of HTML data with applets he fails to teach the use of delay- 
functions within the applet for delaying applet data, instead relying on the interpreter/decoder to display hoth data 
concurrently further using time markers to display data at a predetermined time, thus including attributes to delay 
markup data display , Sam teaches the use of common Java Applet functions that include delay suspending of applet 
data (see pg 2, paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2). He also describes the common applet 
execution states for start, stop and destroy operations using functions (see pg 2, paragraphs 1 & 3). At the time of the 
invention it would have been obvious for the skilled artisan to have implemented the common delay/suspend Java 
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applet functions of Sani with the in\ ention of Puniaveja. The moth at ion for doing so would have been to continuously 
display HTML and applet data concurrently thereby providing information in an integrated seamless package to client 
computers. 

Regarding Dependent claim 16, with dependency of claim 15, Puniaveja teaches wherein the presentation engine 
comprises a buffer to buffer the image output information of the markup document to delay the display of the image 
output information for the markup doctunent, in response to the image output delay signal input from the applet 
executing engine (see abstract & column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically 
lines 34-58, including the explanation provided in the independent claims). 

Regarding Dependent claim 17, with dependency of claim 15, Puniaveja teaches wherein the presentation engine 
comprises an audio buffer, which buffers audio output, and a video buffer, which buffers video output, of the image 
output information of the markup document and/or of the applet output to delay the display of the image output 
information for the markup document, in response to the output control signal input from the applet executing engine 
(see abstract & column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, 
including the explanation provided in the independent claims). 

Regarding Dependent claim 18, with dependency of claim 16, Puniaveja teaches wherein the image output delay 
signal is set according to an amount of rendering time ofthe markup document and/or the applet (see abstract & 
column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the 
explanation provided in the independent claims). 

Regarding Independent claim 19, A computer with a display device to display a markup document image and a 
linked applet image within the markup document image, comprising: a programmed computer processor to control 
synchronous output ofthe markup document image including a linked applet image to the display device, according to 
display control information in the markup document image and or in the linked applet image, so that the markup 
document image and the linked applet image are displaj ed simultaneous!} as a markup image, wherein the applet is 
formed using the Java programming language. 
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Pumaveja teaches the s> nehronized displa} of textual graphical HTML data with Java applets (sec abstract & column 
2, lines 40-55). He teaches the delay of markup image/audio data b> using a decoder which buffers the markup data 
(see fig 9 numerals 966-465 & column 4, lines 5-65, specifically lines 34-58). He states that "Further, since the video 
and annotation streams are handled synchronously but separately by video decoder 964 and annotation interpreter 
963, respectively, steps 1040 and 1050 can occur concurrently... "(see column 9, lines 34-40). Although Pumaveja 
provides a reasonable suggestion lor simultaneous display of HTML data with applets he fails to teach the use of delay 
functions within the applet for delaying applet data, instead relying on the interpreter/decoder to display both data 
concurrently further using time markers to display data at a predetermined time, thus including attributes to delay 
markup data display. Sard teaches the use of common Java Applet functions that include dela\ suspending of applet 
data (see pg 2, paragraphs 1 & 3 & pg 3, paragraphs 1-2 & pg 4, paragraphs 1-2). He also describes the common applet 
execution stales for start, slop antl destro\ operations using functions (sec pg 2, paragraphs 1 & 3). At the time of the 
invention it would have been obvious for the skilled artisan to ha\ c implemented the common dela\ suspend Ja\ a 
applet functions of Sani with the invention of Pumaveja. The motivation for doing so would have been to continuously 
display HTML and applet data concurrently thereby providing information in an integrated seamless package to client 
computers. 

Regarding Dependent claim 20, with dependency of claim 19, Pumaveja teaches wherein the pr gummed computer 
processor controls an order of rendering of the markup document image and the linked applet image according to the 
display control information to s\ nchronousK and simultaneously display the markup document image and the linked 
applet image as the markup image (sec abstract & column 2, lines 40-55 & fig 9 numerals 966-965 & column 9, lines 
5-65, specifically lines 34-58, including the explanation provided in the independent claims). 

Regarding Dependent claim 21, with dependency of claim 19, Pumaveja teaches wherein the display control 
information is used to suspend an output for display of the markup document image until the markup document image 
and the linked applet image are simultaneously displayable as the markup image (see abstract & column 2, lines 40-55 
& fig 9 numerals 966-965 & column 9, lines 5-65, specifically lines 34-58, including the explanation provided hi the 
independent claims). 

// is noted that any citation [fsJJ to specific, pages, columns, lines, or figures in the prior art references and any 
interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it 
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contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. 
[[See, MPEP 2123]] 

Response to Arguments 

6. Applicant's arguments tiled 4/30/2008 have been fully considered but are not persuasive. 

Applicant Argues: l'urna\ cia iuiis lo disc los e or suggest s\ nchronizing the delayed image output information for the 
m ark u p document ". ill) a pplet oulpul inlomKition lor ihe apple t linked lo llie mark up document. Also . I'runax eta tails lo 
disclose or suggest that image output information of the markup document and applet output information of the applet 
are synchronized to be displayed simultaneously. Also Prunaveja fails to disclose or suggest the presentation engine 
that synchronizes and outputs the delayed image output information of the markup document and the applet output to 
the displa\ dc\ ice lor simu ltaneous display. Also Primavera fails to disclose <>r suggest a program med computer 
processor to control synchronous output of the markup document image including the linked applet image lo the 
displa\ de\ Ice, a ccordin g lo ui-.pla\ control in iormalio n inclu d ed in the ma rkup docume nt im age a nd or in ihe linked 
applet image, (pg 6, paragraphs 2-5) 

The Examiner Respectfull) disagrees: Purnaveja teaches the synchronized display of textual graphical HTML data with 
Java applets (see abstract & column 2, lines 40-55). He teaches the delay of markup image/audio data by using a 
decoder which buffers the markup data (see lis c ) numerals 966-965 & column 9, lines 5-65, specifically lines 34-58). 
He states that "Further, since the video and annotation streams are handled synchronously but separately by video 
decoder 964 and annotation interpreter 963, respectively, steps 1 040 and 1 050 can occur concurrently. . ."(see column 
9, lines 34-40). Furthermore the presentation engine is already shown as the video decoders processing the image data 
for display. The invention further includes a computer processor for controlling synchronous display of the image data 
(see ligl ). 

Applicant Argues: Second Sani fails to overcome even the acknowledged deficiency of Purnaveja because Sani fails to 
disclose or suggest delay in g display of image output information for the markup document, (pg 7, paragraph 3) 

The Examiner Respectfully disagrees: Purnaveja already teaches the use of Java applets with HTML pages (a markup 
document) (see abstract) whereas Sani discloses the already well known functions of standard Java applets. 
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In response to applicant's argument, the test for obviousness is not whether the features of a secondary reference may 
be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly 
suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would 
have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). 

It is not necessary that the references actually suggest, expressly or in so many words the changes or improvements that 
applicant has made. The test for combining references is what the references as a whole would have suggested to one of 
ordinary skill in the art. In re Sheckler, 168 USPQ 716 (CCPA 1971); In re McLaughlin 170 USPQ 209 (CCPA 1971); 
In re Young 159 USPQ 725 (CCPA 1968). 

Further more as to the reason to combine not being the same as applicant's. 

If it is obvious to combine references for one reason it is obvious to combine references for all reasons. In re Graf, 145 
USPQ 197 (CCPA 1965); In re Finsterwalder 168 USPQ 530 (USPQ 1970); In re Kronig, 539 F.2d 1300, 190 USPQ 
425 (CCPA 1976). In re Dillon, 892 F.2d 1544, 13 USPQ 1337 (1989); In re Dillon 919 F.2d 688, 16 USPQ 1897 Fed. 
Cir. 1990) (in bane). 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of 
this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1. 136(a) will be calculated from 
the mailing date of the advisoiy action. In no event, however, will the statutory period for reply expire later than SIX MONTHS 
from the mailing dale of this final action. 

Any inquiry concerning this communication or earlier communications from the examiner should be directed to 
Manglesh M. Patel whose telephone number is (571) 272-5937. The examiner can normally be reached on M,F 8:30-6:00 T,TH 
8:30-3:00 Wed 8:30-7:00. 
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If attempts to reach the examiner b\ telephone are unsuccessful, the examiner's supen isor, Stephen S. Hong can be 
reached on (571)272-4124. The lax ph. me number lor the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the Patent Application Information Retrieval 
(PAIR) system. Status information lor published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, 



see blip: /pair-direct. uspto.gox . Should you have questions on ac 


:ess to the Private PAIR system, contact the Electronic Business 


Center (EBC) at 866-217-9197 (toll-free). 
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